Conversation
Adds documentation for the Gemini Interactions API as requested in issue google-gemini#6. Includes overview, quick start examples, best practices, and common use cases.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Summary of ChangesHello @Antovex, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new skill designed to facilitate the development of interactive conversational applications using the Gemini Interactions API. It provides structured guidance and best practices for managing conversation state and turn-based interactions, enhancing the project's capabilities for building sophisticated chat experiences. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new skill for the Gemini Interactions API, which is a valuable addition. The documentation provides a good overview, but the quick-start code examples are currently just placeholders. My review includes suggestions to replace these placeholders with complete, minimal, and working examples for both Python and JavaScript/TypeScript. These changes will make the skill much more useful for developers looking to get started with interactive chat experiences.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
skills/interactions-api/SKILL.md
Outdated
|
|
||
| ### Python | ||
| ```python | ||
| import google.generativeai as genai |
There was a problem hiding this comment.
One of the main drivers for this skill is to stop LLMs from using this deprecated SDK. I don't mind if you use an LLM to bootstrap the skill but please 1) test it and 2) use the gemini-api-dev skill.
There was a problem hiding this comment.
I have tested all the example code locally and have referenced the gemini-api-dev skill. I have made few commits regarding the same. please check it out and let me know if any changes are required.
…ss multi-turn examples
… add-interactions-api-skill
|
Hi @markmcd, thanks for the review and the callout about the deprecated SDK. |
|
I need to see the matching files first |
@Vikxlife which files are you talking about ? |
|
Awesome thank you @Antovex. Hey @philschmid would you mind taking a look at this too? |
|
Hi @philschmid , do let me know if there are any changes required. |
skills/interactions-api/SKILL.md
Outdated
|
|
||
| ### Python | ||
| ```python | ||
| import google.generativeai as genai |
There was a problem hiding this comment.
Can we update this to match the official docs.
There was a problem hiding this comment.
Will include the same imports as official documentation
| - **State Management** - Maintain conversation context across interactions (stateful or stateless) | ||
| - **Turn-based Communication** - Structured request/response patterns | ||
| - **Session Handling** - Manage conversation sessions efficiently with interaction IDs | ||
| - **Multimodal Support** - Process text, images, audio, video, and documents |
There was a problem hiding this comment.
Should include tool use capabilties and agent support
There was a problem hiding this comment.
Mentioning tool and function calling here.
skills/interactions-api/SKILL.md
Outdated
|
|
||
| ## Resources | ||
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). |
There was a problem hiding this comment.
Agent don't have context about ../ links
There was a problem hiding this comment.
So for now I am giving the reference for this file using the full github path to that file, but if it needs to be changes do let me know.
skills/interactions-api/SKILL.md
Outdated
| ## Resources | ||
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) |
There was a problem hiding this comment.
We should only have the .md version
skills/interactions-api/SKILL.md
Outdated
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) |
There was a problem hiding this comment.
Agents cannot access easilly colab, we should remove
There was a problem hiding this comment.
removing the mention of colab.
skills/interactions-api/SKILL.md
Outdated
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) | ||
| - [API Reference](https://ai.google.dev/api/interactions-api) |
skills/interactions-api/SKILL.md
Outdated
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) | ||
| - [API Reference](https://ai.google.dev/api/interactions-api) | ||
| - [Gemini API Documentation](https://ai.google.dev/gemini-api/docs/llms.txt) |
There was a problem hiding this comment.
Changing it to "Gemini API llms.txt" for now, but if something better can be used for this, do let me know.
skills/interactions-api/SKILL.md
Outdated
| ## Best Practices | ||
|
|
||
| 1. **Use Stateful Conversations**: Leverage `previous_interaction_id` for automatic state management instead of manually tracking history | ||
| 2. **Handle Storage**: Interaction objects are saved by default (`store=true`). Be mindful of data retention policies | ||
| 3. **Error Handling**: Implement robust error handling for network and API issues | ||
| 4. **Rate Limiting**: Be mindful of API rate limits in interactive scenarios | ||
| 5. **Beta Considerations**: The API is in Beta and subject to breaking changes |
There was a problem hiding this comment.
Doesn't feel helpful for using the skill and more general developer principles should live outside.
There was a problem hiding this comment.
Ok, then i will totally remove this.
| - Multi-turn chatbots with context awareness | ||
| - Interactive assistants with session management | ||
| - Conversational interfaces with multimodal inputs | ||
| - Context-aware Q&A systems | ||
| - Document analysis and understanding | ||
| - Image, audio, and video processing workflows |
skills/interactions-api/SKILL.md
Outdated
|
|
||
| --- | ||
|
|
||
| For more details, refer to the [official documentation](https://ai.google.dev/gemini-api/docs/interactions). No newline at end of file |
There was a problem hiding this comment.
Should be markdown link or removed as it is part further up
Antovex
left a comment
Antovex
left a comment
There was a problem hiding this comment.
Soon there will be a commit with all the changes mentioned, do check it and let me know if there is any refactoring required.
| - **State Management** - Maintain conversation context across interactions (stateful or stateless) | ||
| - **Turn-based Communication** - Structured request/response patterns | ||
| - **Session Handling** - Manage conversation sessions efficiently with interaction IDs | ||
| - **Multimodal Support** - Process text, images, audio, video, and documents |
There was a problem hiding this comment.
Mentioning tool and function calling here.
skills/interactions-api/SKILL.md
Outdated
|
|
||
| ### Python | ||
| ```python | ||
| import google.generativeai as genai |
There was a problem hiding this comment.
Will include the same imports as official documentation
skills/interactions-api/SKILL.md
Outdated
|
|
||
| ## Resources | ||
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). |
There was a problem hiding this comment.
So for now I am giving the reference for this file using the full github path to that file, but if it needs to be changes do let me know.
skills/interactions-api/SKILL.md
Outdated
| ## Resources | ||
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) |
skills/interactions-api/SKILL.md
Outdated
|
|
||
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) |
There was a problem hiding this comment.
removing the mention of colab.
skills/interactions-api/SKILL.md
Outdated
| - See the [gemini-api-dev skill](../gemini-api-dev/SKILL.md) for canonical SDK guidance (installation, migration notes, and recommended models such as the google-genai / @google/genai SDKs). | ||
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) | ||
| - [API Reference](https://ai.google.dev/api/interactions-api) |
skills/interactions-api/SKILL.md
Outdated
| - [Official Interactions API Documentation](https://ai.google.dev/gemini-api/docs/interactions) | ||
| - [Interactions API Quickstart Notebook](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_interactions_api.ipynb) | ||
| - [API Reference](https://ai.google.dev/api/interactions-api) | ||
| - [Gemini API Documentation](https://ai.google.dev/gemini-api/docs/llms.txt) |
There was a problem hiding this comment.
Changing it to "Gemini API llms.txt" for now, but if something better can be used for this, do let me know.
skills/interactions-api/SKILL.md
Outdated
| ## Best Practices | ||
|
|
||
| 1. **Use Stateful Conversations**: Leverage `previous_interaction_id` for automatic state management instead of manually tracking history | ||
| 2. **Handle Storage**: Interaction objects are saved by default (`store=true`). Be mindful of data retention policies | ||
| 3. **Error Handling**: Implement robust error handling for network and API issues | ||
| 4. **Rate Limiting**: Be mindful of API rate limits in interactive scenarios | ||
| 5. **Beta Considerations**: The API is in Beta and subject to breaking changes |
There was a problem hiding this comment.
Ok, then i will totally remove this.
| - Multi-turn chatbots with context awareness | ||
| - Interactive assistants with session management | ||
| - Conversational interfaces with multimodal inputs | ||
| - Context-aware Q&A systems | ||
| - Document analysis and understanding | ||
| - Image, audio, and video processing workflows |
skills/interactions-api/SKILL.md
Outdated
|
|
||
| --- | ||
|
|
||
| For more details, refer to the [official documentation](https://ai.google.dev/gemini-api/docs/interactions). No newline at end of file |
Added sections on tool use and function calling to the Interactions API documentation. Updated best practices and resources for better guidance.
|
I have an idea: is the name
|
This is a good idea !! Would do if any of the reviewers or maintainers approve it. |
do you work remotely? @Antovex ? |
Yes, I do. Could you clarify what you mean by 'remotely' in this context? |
5 participants
Fixes #6
This PR adds a new skill for the Gemini Interactions API as requested.
Changes:
skills/interactions-api/SKILL.mdwith documentation about the Interactions API